Roof plane segmentation by combining multiple images and point clouds

A new method for roof plane detection using multiple aerial images and a point cloud is presented. It takes advantage of the fact that segmentation results for different views look different even if the same parameters are used for the original segmentation algorithm. The point cloud can be generated by image matching or by airborne laserscanning. Plane detection starts by a segmentation that is applied to each of the images. The point cloud is used to determine which image segments correspond to planes. The best plane according to a criterion is selected and matched with segments in the other images. Matching of segments requires a DSM generated from the point cloud, and it takes into account the occlusions in each image. This procedure is repeated until no more planes can be found. After that, planar segments are extracted based on region growing in the point cloud in areas of severe under-segmentation, and the multiple-image segmentation procedure is repeated. Finally, neighbouring regions found to be co-planar are merged. First results are presented for test site with up to nine-fold overlap. Our tests show that the method can deliver a good separation of roof planes under difficult circumstances, though the level of detail that can be achieved is limited by the resolution of the point cloud

Classification of settlement areas in remote sensing imagery using conditional random fields

Land cover classification plays a key role for various geo-based applications. Numerous approaches for the classification of settlements in remote sensing imagery have been developed. Most of them assume the features of neighbouring image sites to be conditionally independent. Using spatial context information may enhance classification accuracy, because dependencies of neighbouring areas are taken into account. Conditional Random Fields (CRF) have become popular in the field of pattern recognition for incorporating contextual information because of their ability to model dependencies not only between the class labels of neighbouring image sites, but also between the labels and the image features. In this work we investigate the potential of CRF for the classification of settlements in high resolution satellite imagery. To highlight the power of CRF, tests were carried out using only a minimum set of features and a simple model of context. Experiments were performed on an Ikonos scene of a rural area in Germany. In our experiments, completeness and correctness values of 90% and better could be achieved, the CRF approach was clearly outperforming a standard Maximum-Likelihood-classification based on the same set of features

Using building and bridge information for adapting roads to ALS data by means of network snakes

In the German Authoritative Topographic Cartographic Information System (ATKIS), the 2D positions and the heights of objects such as roads are stored separately in the digital landscape model (DLM) and digital terrain model (DTM), which is often acquired by airborne laser scanning (ALS). However, an increasing number of applications require a combined processing and visualization of these two data sets. Due to different kinds of acquisition, processing, and modelling discrepancies exist between the DTM and DLM and thus a simple integration may lead to semantically incorrect 3D objects. For example, roads may be situated on strongly tilted DTM parts and rivers sometimes flow uphill. In this paper we propose an algorithm for the adaptation of 2D road centrelines to ALS data by means of network snakes. Generally, the image energy for the snakes is defined based on ALS intensity and height information and derived products. Additionally, buildings and bridges as strong features in height data are exploited in order to support the road adaptation process. Extracted buildings as priors modified by a distance transform are used to create a force of repulsion for the road vectors integrated in the image energy. In contrast, bridges give strong evidence for the correct road position in the height data. Therefore, the image energy is adapted for the bridge points. For that purpose bridge detection in the DTM is performed starting from an approximate position using template matching. Examples are given which apply the concept of network-snakes with new image energy for the adaptation of road networks to ALS data taking advantage of the prior known topology

Deep Descriptor Learning with Auxiliary Classification Loss for Retrieving Images of Silk Fabrics in the Context of Preserving European Silk Heritage

With the growing number of digitally available collections consisting of images depicting relevant objects from the past in relation with descriptive annotations, the need for suitable information retrieval techniques is becoming increasingly important to support historians in their work. In this context, we address the problem of image retrieval for searching records in a database of silk fabrics. The descriptors, used as an index to the database, are learned by a convolutional neural network, exploiting the available annotations to automatically generate training data. Descriptor learning is combined with auxiliary classification loss with the aim of supporting the clustering in the descriptor space with respect to the properties of the depicted silk objects, such as the place or time of origin. We evaluate our approach on a dataset of fabric images in a kNN-classification, showing promising results with respect to the ability of the descriptors to represent semantic properties of silk fabrics; integrating the auxiliary loss improves the overall accuracy by 2.7% and the average F1 score by 5.6%. It can be observed that the largest improvements can be obtained for variables with imbalanced class distributions. An evaluation on the WikiArt dataset demonstrates the transferability of our approach to other digital collection

Automatic road network extraction in suburban areas from high resolution aerial images

In this paper a road network extraction algorithm for suburban areas is presented. The algorithm uses colour infrared (CIR) images and digital surface models (DSM). The CIR data allow a good separation between vegetation and roads. The image is first segmented in two steps: an initial segmentation using the normalized cuts algorithm and a subsequent grouping of the segments. Road parts are extracted from the segments and then first connected locally to form subgraphs, because roads are often not extracted as a whole due to disturbances in their appearance. Subgraphs can contain several branches, which are resolved by a subsequent optimisation. The optimisation uses criteria describing the relations between the road parts as well as context objects such as trees, vehicles and buildings. The resulting road strings, represented by their centre lines, are then connected to a road network by searching for junctions at the ends of the roads. Small isolated roads are eliminated because they are likely to be false extractions. Results are presented for three image subsets coming from two different data sets, and a quantitative analysis of the completeness and correctness is shown from nine image subsets from the two data sets. The results show that the approach is suitable for the extraction of roads in suburban areas from aerial images

Detection and 3D modelling of vehicles from terrestrial stereo image pairs

The detection and pose estimation of vehicles plays an important role for automated and autonomous moving objects e.g. in autonomous driving environments. We tackle that problem on the basis of street level stereo images, obtained from a moving vehicle. Processing every stereo pair individually, our approach is divided into two subsequent steps: the vehicle detection and the modelling step. For the detection, we make use of the 3D stereo information and incorporate geometric assumptions on vehicle inherent properties in a firstly applied generic 3D object detection. By combining our generic detection approach with a state of the art vehicle detector, we are able to achieve satisfying detection results with values for completeness and correctness up to more than 86%. By fitting an object specific vehicle model into the vehicle detections, we are able to reconstruct the vehicles in 3D and to derive pose estimations as well as shape parameters for each vehicle. To deal with the intra-class variability of vehicles, we make use of a deformable 3D active shape model learned from 3D CAD vehicle data in our model fitting approach. While we achieve encouraging values up to 67.2% for correct position estimations, we are facing larger problems concerning the orientation estimation. The evaluation is done by using the object detection and orientation estimation benchmark of the KITTI dataset (Geiger et al., 2012).DFG/GRK/215

Invariant descriptor learning using a Siamese convolutional neural network

In this paper we describe learning of a descriptor based on the Siamese Convolutional Neural Network (CNN) architecture and evaluate our results on a standard patch comparison dataset. The descriptor learning architecture is composed of an input module, a Siamese CNN descriptor module and a cost computation module that is based on the L2 Norm. The cost function we use pulls the descriptors of matching patches close to each other in feature space while pushing the descriptors for non-matching pairs away from each other. Compared to related work, we optimize the training parameters by combining a moving average strategy for gradients and Nesterov's Accelerated Gradient. Experiments show that our learned descriptor reaches a good performance and achieves state-of-art results in terms of the false positive rate at a 95% recall rate on standard benchmark datasets

The Automatic Extraction of Roads from LIDAR data

A method for the automatic detection of roads from airborne laser scanner data is presented. Traditionally, intensity information has not been used in feature extraction from LIDAR data because the data is too noisy. This article deals with using as much of the recorded laser information as possible thus both height and intensity are used. To extract roads from a LIDAR point cloud, a hierarchical classification technique is used to classify the LIDAR points progressively into road or non-road. Initially, an accurate digital terrain model (DTM) model is created by using successive morphological openings with different structural element sizes. Individual laser points are checked for both a valid intensity range and height difference from the subsequent DTM. A series of filters are then passed over the road candidate image to improve the accuracy of the classification. The success rate of road detection and the level of detail of the resulting road image both depend on the resolution of the laser scanner data and the types of roads expected to be found. The presence of road-like features within the survey area such as private roads and car parks is discussed and methods to remove this information are entertained. All algorithms used are described and applied to an example urban test site